• Dose Escalation additional criteria: Patients must have histologically confirmed cancer that is metastatic or unresectable and must have progressed on standard therapies which would have included fluorouracil (5-FU) or capecitabine

• Dose Escalation additional criteria specifically for colorectal cancer (CRC) patients: Willingness and ability to undergo a pre-treatment biopsy

• Dose Expansion additional criteria: Patients must have histologically confirmed CRC that is metastatic or unresectable and must have progressed on standard therapies which would have included 5-FU or capecitabine

• Dose Expansion additional criteria: Willingness and ability to undergo pre- and on- treatment biopsies

• Patients must have measurable disease

• Age \>= 18 years. Because no dosing or adverse event data are currently available on the use of ZEN003694 (ZEN-3694) in combination with capecitabine in patients \< 18 years of age, children are excluded from this study

• Eastern Cooperative Oncology Group (ECOG) performance status of 0-2 (Karnofsky \>= 60%)

• Availability of archival tumor tissue at the time of patient enrollment for molecular profiling studies

• Prior to study dosing, previous systemic therapy must have been completed for at least five half-lives or 2 weeks, whichever is shorter

• Absolute neutrophil count \>= 1,000/mcL

• Platelets \>= 100,000/mcL

• Total bilirubin =\< 1.5 institutional upper limit of normal (ULN)

• Aspartate aminotransferase (AST) (serum glutamic-oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 3 x institutional ULN

• Glomerular filtration rate (GFR) \>= 50 mL/min/1.73 m\^2

• Human immunodeficiency virus (HIV)-infected patients on effective anti-retroviral therapy with undetectable viral load within 6 months are eligible for this trial

• For patients with evidence of chronic hepatitis B virus (HBV) infection, the HBV viral load must be undetectable on suppressive therapy, if indicated

• Patients with a history of hepatitis C virus (HCV) infection must have been treated and cured. For patients with HCV infection who are currently on treatment, they are eligible if they have an undetectable HCV viral load

• Patients with treated brain metastases are eligible if follow-up brain imaging after central nervous system (CNS)-directed therapy shows no evidence of progression

• Patients with a prior or concurrent malignancy whose natural history or treatment does not have the potential to interfere with the safety or efficacy assessment of the investigational regimen are eligible for this trial

• Patients should be New York Heart Association Functional Classification of class 2B or better

• The effects of ZEN003694 (ZEN-3694) and capecitabine on the developing human fetus are unknown. For this reason and because BET inhibitors as well as other therapeutic agents used in this trial are known to be teratogenic, women of child-bearing potential and men must agree to use adequate contraception (hormonal or barrier method of birth control; abstinence) prior to study entry and for the duration of study participation. Women of child-bearing potential and men treated or enrolled on this protocol must also agree to use adequate contraception prior to the study, for the duration of study participation, and 6 months after completion of ZEN003694 (ZEN-3694) and capecitabine administration

• Ability to understand and the willingness to sign a written informed consent document. Legally authorized representatives may sign and give informed consent on behalf of study participants